Human-Machine Hybrid Peer Grading in SPOCs

Human-Machine Hybrid Computation

INTRODUCTION A common thread among fields such natural language processing, computer vision, and artificial intelligence is that all seek to automate tasks which humans do naturally. Examples include listening to human speech, seeing and recognizing objects, reading text, and understanding meaning. If they could be automated by computers, then they could be done on demand, and much more quickly...

How Accurate Is Peer Grading?

Previously we showed that weekly, written, timed, and peer-graded practice exams help increase student performance on written exams and decrease failure rates in an introductory biology course. Here we analyze the accuracy of peer grading, based on a comparison of student scores to those assigned by a professional grader. When students graded practice exams by themselves, they were significantl...

Incentives for Truthful Peer Grading

Peer grading systems work well only if users have incentives to grade truthfully. An example of nontruthful grading, that we observed in classrooms, consists in students assigning the maximum grade to all submissions. With a naive grading scheme, such as averaging the assigned grades, all students would receive the maximum grade. In this paper, we develop three grading schemes that provide ince...

Recent Advances in Large-Scale Peer Grading

Dynamics of Peer Grading: An Empirical Study

Peer grading is widely used in MOOCs and in standard university settings. The quality of grades obtained via peer grading is essential for the educational process. In this work, we study the factors that influence errors in peer grading. We analyze 288 assignments with 25,633 submissions and 113,169 reviews conducted with CrowdGrader, a web based peer grading tool. First, we found that large gr...